The number of electronic images being created is increasing at a rapid rate, and searching them semantically presents a significant challenge. Many raw images are made available with few meaningful direct annotations of semantic content, limiting their search and discovery. While some image repositories or Web sites encourage tags or keywords to be included manually, such is far from universal. Manual characterization of semantic image contents is often subjective, labor intensive, and inconsistent. The relative time-consuming process of manual characterization or annotation is also unlikely to be able to keep up with the rate of creation of images through digital and non-digital means.
There have been attempts in the imaging art to characterize image contents. For example, U.S. Pat. No. 7,555,165 relates to a method for semantic scene characterization using camera data and content-based cues. However, this patent does not use GPS data, time information nor image-recognition techniques such as face detection, face recognition which limits the amount of information that can be determined about an unknown image. This patent also is limited to placing images into classes and does not provide annotations which are richer and more versatile.
Published U.S. Patent Application 2009-0222432 relates to geo-tagging and automatic generation of metadata for photos and videos. Although the use of GPS coordinates is helpful in the classification of images, it does not provide the means to characterize the content of images.
IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 30, No. 11, pp. 1933-1944, November, 2008, authored by the present inventors and the disclosure of which is incorporated by reference herein, discusses semantic annotation of images using metadata extraction. Color feature extraction, shape feature extraction, and texture feature extraction are further applied to images to create a semantic database of images.
However, there remains a need in the art for improved annotation and classification of semantic image contents. For example, using the present invention, an image may be automatically annotated as “Jack Kennedy by the sea at sunset in Gold Coast, Australia on the eve of the Indian Ocean Tsunami.”, which is not possible with the above-mentioned art.